Electronic Design Automation (EDA) tools are computer software programs used to design electronic circuits. A suite of EDA tools may be used to convert a circuit design idea to a physical design including logic gates, circuit components and their interconnections. A circuit designer typically uses an EDA tool to create a schematic design of a circuit. The designer may use other EDA tools from the suite to convert the schematic to a physical layout, which specifies the shapes, positions, and dimensions of semiconductor and conductor materials such as silicon and metal. The EDA design tools can be used to in essence, translate a schematic design to a physical layout using layout rules, which specify, for example, that a particular type of transistor is to have particular dimensions and be a particular distance from other transistors in the physical layout. The physical layout can be fabricated to produce a physical implementation, which typically involves a semiconductor integrated circuit (IC) chip or printed circuit board (PCB). The integrated circuit may be an analog circuit such as an amplifier, a digital circuit such as a microprocessor, or a combination analog and digital circuit, such as a cellular telephone radio, for example.
In one approach to circuit design, a designer creates a visual representation of a circuit design on a design canvas. A design canvas typically comprises a computer generated two-dimensional interactive user interface screen display area produced with the help of an EDA tool. The EDA tool encodes a computer system to produce the user interface that allows a designer to create such a schematic diagram by adding representations of design objects, which comprise data code structures encoded in a computer readable medium that correspond to graphical icons that represent physical circuit elements such as, transistors, resistors, digital logic gates, and other devices to the design canvas. The EDA tool user interface (UI) also may allow the designer to change an existing schematic design by modifying existing design objects on the canvas, by adding new design objects to the canvas, and by modifying or adding connections between the design objects on the canvas. Connections between design objects in the schematic generally represent physical electrical connections between circuit elements of a circuit under development.
User interactions with an EDA tool UI can be complex, and there have been efforts to improve user productivity through user training and UI improvement. These efforts typically involve gathering information to identify opportunities for improving user productivity based upon how users actually interact with the EDA tool UI. In particular, log files are produced to record user interaction with the EDA tool UI. In general, a ‘log file’ records events that occur in a computer system. Some example events include “call”, “execute”, “open”, “close”, “wait”, “fetch”, “sort”, “append”, “run”, and “delete”. A log file typically comprises a text listing of events in chronological order in which the events occur in a computer system. A log file ordinarily contains information about source code but not the source code itself. Thus, different log file entries represent different configurations of a computer system used to process different user commands that a user inputs via a UI over some time frame. In practice, hundreds to thousands of session log files may be gathered for analysis.
In the past, a log miner software tool was employed to evaluate information in a log file. Often, the log miner was programmed to search for certain regular expressions that typically appear as simple strings patterns indicative of user UI interactions. A frequent ‘itemset’ analysis often was performed to determine which low level tool commands, error messages, diagnostics, etc., occur most frequently. Based upon this information, users could be re-trained when, for example, an analysis of user interaction with the UI indicated that users are not using some desirable new productivity-enhancing features due to being stuck in their ‘old ways’. Alternatively, based upon the evaluation, the EDA tool software could be modified to better adapt it to existing use patterns (e.g. make most commonly used options available in the GUI, to hide less commonly used options or to make buttons that are often used in tandem available close together on the screen)
While frequency analysis techniques glean useful information, more complex questions require more in depth analysis to answer. For example, the following question might be posed, “how often do users attempt to do activity ‘ABC’, but are foiled in their attempts encountering a particular type (or types) of error message?” While it is relatively easy for us to measure how often a particular type of error message appears, it is not so easy to know when it appears in the context of activity ‘ABC’, as opposed to in the context of some other activity ‘PQR’, for example.
Moreover, line-by-line inspection of a log file entries to evaluate the occurrence of some event may be difficult since a log file text may be somewhat amorphous. For example, the question may be asked, “how often do users perform action ‘XYZ’?” In some cases, action ‘XYZ’ may actually correspond to a contiguous sequence of low level log file entries, but in practice, these entries often are interspersed with system diagnostics, and other log file entries which are placed there for other purposes not related to this mining activity (e.g. for test replay purposes). Such entries lead to the text logs being somewhat amorphous, i.e. not as nicely structured and easily comprehensible as one would like (or even expect), making it difficult answer such questions using a simple line-by-line inspection approach.
FIGS. 1A-1G are simplified illustrative drawings representing a sequence of screen displays produced by an electronic design automation (EDA) tool user interface in response to a sequence of user input commands. In this illustrative example, the screen displays show portions of an integrated circuit design layout A02. Each rectangular region labeled 1-12 represents a different ‘geographical’ region of the design layout. It will be appreciated that each region may contain numerous structural details of the circuit design, which are not shown since these details are not relevant to the description. The rectangle labeled X represents a portion of the circuit design that is visible within the display screen. Thus, the rectangle X represents the screen display. In FIG. 1A, portions of regions 1, 2, 5, and 6 are visible within the screen display X. Starting with FIG. 1A, a user inputs UI commands to cause the screen display to pan downward so that regions 5, 6, 9 and 10 are visible within the screen display X as shown in FIG. 1B. Next, a user inputs UI commands to cause the screen image to pan right so that regions 6, 7, 10 and 11 are visible within the screen display X as shown in FIG. 1C. Next, the user inputs UI commands to cause regions 6 and 7 of the design to be deleted as shown in FIG. 1D. Next, the user inputs UI commands to cause regions 2 and 3 to move to the portion of the design screen display X that were emptied due to the removal of regions 6 and 7 as shown in FIG. 1E. Next, the user inputs UI commands to undo the move operation resulting in the return of regions 2 and 3 to their prior locations, as shown in FIG. 1F. Next, the user inputs UI commands to undo the delete operation to cause regions 6 and 7 to be returned to the portion of the design that they were deleted from as shown in the screen display X of FIG. 1G.
Table 1 shows an illustrative portion of a log file that may be produced for the delete, move, undo, undo operations of FIGS. 1D-1G.
TABLE 1\a hiZoomIn( )\i 135.01.192.62\p >\a addPoint(hiGetCommandPoingt( ))\i 199.01.132.415\r t\t t
In some embodiments, log files are created that are indicative of user interaction with the user interface to a computer system configured to implement a computer software based EDA tool. FIG. 2 shows illustrative log files that are structured into different kinds of uniquely-prefixed entries. The example EDA tool can replay a session given a log file from that session. The prefixes are used by a computer system to distinguish between different kinds of information in a log file when replaying a session from the log file information. Replaying a session is useful, for example, to reproduce a bug or to automatically test software.
Table 2 provides a description of the different types of chunk identifying entries and of the different prefixes that are associated with the different kinds of UI event entries.
TABLE 2PrefixPrefix Meaning\aThese lines reflect interactive user entries i.e. the commands invoked due to menu picks, etc.\iThese lines represent additional user input (e.g. coordinates of amouse click within a layout window)\rThese lines represent system responses (typically status flags like t representing true, and nil representing False, per the ‘SKILL’ computer program language syntax)\pThese lines represent system-generated prompts to the user\oThese lines contain system-generated output/informational messages\wThese lines contain system-generated warning messages\e These lines correspond to system-generated errors\#lines correspond to system-generated diagnostics (such as memoryusage, etc), more generally intended for statistical mining than foruser consumption
Referring again to Table 1, log file portion (7 lines) within region shows the log file entries corresponding to a simple user Zoom in (via a ‘rubberband’ command). The entry, ‘hiZoomIn( )’ indicates that a user invokes a zoom command; the prefix ‘\a’ indicates that the entry is an interactive user entry. The entry ‘135.01:192.62’ indicates that a user selects (e.g. ‘clicks’) to establish the top left corner of the bounding box in circuit design database coordinate system of the rubberband/bounding box for the zoom; the prefix ‘\i’ indicates that the entry is an additional user input. The entry ‘>’ indicates a computer system prompt to the user to select (e.g. ‘click’) to establish the bottom right corner of the bounding box; the prefix ‘\p’ indicates a system-generated prompt to the user. The entry ‘addPoint(hiGetCommandPoint( )) indicates a user command to select (e.g. ‘clicks’) to furnish a bottom right corner of the bounding box; the prefix ‘\a’ indicates that the entry is an interactive user entry. The entry ‘199.01:132.415’ indicates that a user selects (e.g. ‘clicks’) to establish the bottom right corner of the bounding box in circuit design database coordinate system; the prefix ‘\i’ indicates that the entry is an additional user input. The entries ‘t’ represent the computer system response: the prefix ‘\r’ indicates a system response. It will be appreciated that the entire log file portion (7 lines) may be contained within a larger overall log file information that is contained within a buffer storage device.
Log file text sequences of Table 1 could potentially be identified by comparison with ‘regular expressions’ such as the PCRE (Perl Compatible Regular Expressions), for example. However, in practice such expressions can turn out to be extremely long, difficult to code in a robust manner, and may contain ambiguities that can be difficult to debug. The difficulty increases when trying to code robust regular expressions (‘regexps’) that can handle varied interspersed warnings, prompts, user inputs, etc., within the log file. Often, the interspersed data turns out to be seemingly randomly interspersed due to asynchronous events within the computer system (related to inter-process-communication aka IPC-related activities). Sometimes information concerning such events is present in the log file, and other times it is not. In other words, there may be no easily discernable pattern to these intervening events, i.e. the pattern of events is not necessarily all that regular. Extracting useful information concerning user interaction in spite of amorphisms such as these intervening events is a challenge to mining in the log file.
Moreover, for analysis of some UI events involving user interactions/experiences with a UI, log entries relating to computer system actions (such as system response entries, or user prompt entries, or error/warning/info entries) may be irrelevant to evaluating some user actions/experiences with a UI, but may be important to evaluating other user actions/experiences. For example, one user activity of interest may involve looking for a particular error message or warning within a particular sequence of log entries, (i.e. within a certain context). However, such error messages may be irrelevant to evaluation of other user activities, and would constitute irrelevant information or ‘noise’.